Skip to content

feat(e2e): run the CLI on Bun 1.4.3 and newer, without Node.js - #1616

Merged
okwasniewski merged 12 commits into
mainfrom
oskar/bun-runtime-support
Nov 9, 2026
Merged

okwasniewski merged 12 commits into
mainfrom
oskar/bun-runtime-support

Conversation

@okwasniewski

@okwasniewski okwasniewski commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

The e2e CLI runs on Bun 1.4.3 or newer with no Node.js installed: bunx --bun e2e run. Node.js behavior does not change.

What changed

  • Loader. e2e's Node.js loader runs on module.registerHooks, which Bun lacks (oven-sh/bun#35690 is a draft). On Bun, e2e registers no hooks. Bun compiles TypeScript and resolves tsconfig paths and # imports itself (config/bun-loader.ts).
  • Fresh instances. Bun ignores the query e2e uses on Node.js to key module instances. So the loader deletes the module from require.cache, which holds ES modules too, and imports it again. A fresh graph also reloads every project file the entry imports by path, # import, or tsconfig alias. Bun.Transpiler.scanImports and Bun.resolveSync find those files.
  • Redaction. Bun's console writes past the process.stdout.write that e2e patches to redact secrets and route output. The loader points console at the streams before any project code runs.
  • One run per process. Bun runs an imported module's top level outside its importer's AsyncLocalStorage context, and holds one instance of each module per process. Two runs in one process can't be kept apart, so on Bun:
    • e2e/runner throws NODE_UNSUPPORTED. It already did, because of the missing hooks.
    • e2e mcp opens one session at a time.
  • Version guard. A Bun older than 1.4.3 gets the version to upgrade to: Bun 1.4.2 races a require() of an ES module whose graph waits on top-level await. Deno is still refused.
  • Fixes Bun exposed:
    • The signal ladder trusts process.connected only when there is an IPC channel (cli/signals.ts).
    • Stack frames may have no column (collect/registry.ts).
    • Config errors read Bun's syntax and resolve error shapes (config/diagnose.ts).
  • Init. In a Bun project, e2e init registers the MCP server with bunx.
  • CI. A bun job runs tests/integration/bun-runtime.test.ts on Bun 1.4.3 and on the latest Bun. The CLI, and everything it spawns, gets a PATH with no node on it.
flowchart LR
  L[load a config or test file] --> R{runtime}
  R -- Node.js --> N["registerHooks + oxc<br/>an instance per ?e2e-graph= query"]
  R -- Bun --> B["Bun compiles it<br/>delete from require.cache, import again"]
Loading

CLI reference, Bun lists what still differs on Bun:

  • # imports that map to a directory.
  • Legacy decorators in .cts files.
  • Watch mode and MCP read tsconfig.json and computed-path imports one time.
  • executable: 'node' under bun <file>.
  • The line reported for a test that a helper declares after an await.

The mobile engine and the integrations are not verified on Bun yet.

Verified

Ran it locally: yes

  • Released e2e vs this branch: e2e --version on Bun 1.4.4, with only Bun and git on PATH.

    e2e 0.23.0 from npm this branch
    e2e runs on Node.js: Bun 1.4.4 reports Node.js 26.3.0 but has no module.registerHooks, which e2e's TypeScript loader needs. (exit 2) 0.23.0. Then e2e doctor on the testbed prints ✓ node Bun 1.4.4 (>=1.4.3) and e2e run can start here
  • Testbed on Bun: the built CLI on Bun 1.4.4, run with env -i PATH=<bun and git>:/usr/bin:/bin bun run e2e run. Same totals as on Node.js 24.19:

     Test Files  47 passed (47)
          Tests  587 passed | 1 skipped (588)
       Duration  23.41s
    
  • e2e mcp --max-sessions 3 on Bun: it logs that the flag is ignored. A second open_session gets no session slot is free (e2e mcp --max-sessions 1).

  • Reverting fixes: each Bun test added for a loader or MCP fix fails with its fix reverted. That covers reloading a JSX .js file and a data file in a fresh graph, query imports, dotted aliases, console redaction, and the one-session cap.

View guided diff Turn on auto-fix

Moves the loader project fixture to tests/helpers/typescript-project.ts so the Bun suite reuses it.
Bun sets process.connected to false without a channel, so the CLI took a missing parent for a dead one and interrupted its own run at once. process.send marks a process spawned with a channel.
npx needs Node.js; a Bun project may have none.
Bun compiles the project's TypeScript itself, so e2e registers no module
hooks there. What Bun lacks is a fresh instance: it drops the import query
e2e keys instances by, so a re-imported test file registered no tests.
config/bun-loader.ts deletes the module from Bun's registry (require.cache
holds ES modules too) before loading it again, follows a fresh graph's
project imports with Bun.Transpiler.scanImports and Bun.resolveSync, runs
one load at a time, and evaluates with require() so a config's top level
keeps its load's async context; top-level await finishes under import().

- internal/runtime.ts (was node-version.ts): Bun floor 1.4.3; 1.4.2 races
  require() of an ES module in a graph waiting on top-level await
- registry: read JavaScriptCore frames without a column, skip `native`
- diagnose: Bun BuildMessage positions, # import and missing export hints
- doctor and report name Bun; telemetry shares runtimeOf
- tests/integration/bun-runtime.test.ts runs the built CLI on Bun with no
  node on PATH; spec.yml runs it on Bun 1.4.3 and latest
From the GPT 6 Astra review of the Bun branch, each reproduced on Bun
1.4.3 and 1.4.4:

- Bun's console wrote past the patched process.stdout, so console.log
  printed secrets verbatim and broke --reporter json. bun-loader now
  points console at the redacted streams before any project module runs.
- Bun runs an imported module's top level outside its importer's async
  context, require() included once a dependency awaits. e2e/runner keeps
  runs apart by that context, so it refuses Bun (NODE_UNSUPPORTED), and
  e2e mcp holds one session on Bun. With nothing context-bound left,
  loads use plain import(): require-first and its fallback go.
- One module instance per file means one live graph: a single current
  graph replaces the per-name map.
- Alias classification asks Bun to resolve each tsconfig candidate; the
  stem heuristic missed dotted names (@lib/value.config) and left them
  stale. Queried imports keep their query in the module id only.
- An explicit E2E_BUN runs the support tests on any Bun, so a moved floor
  fails CI; a fixture now pins the 1.4.2 require() race.
The Deno and missing-registerHooks branches protect no Node.js or Bun
case: Bun returns before them, and every supported Node.js has the hooks.
runtimeOf had two callers left, so doctor and report name the runtime
inline, and telemetry keeps the detection it had before this branch.
serverEntry had one caller, and cli-init.test.ts already checks the bunx
entry. The changeset, CI intro, and setup skill now say what users need
and link the Bun limitations.
…fusing Deno

- Bun loader scans .js files as JSX, as Bun runs them, and reloads data files (.toml, .txt, ...); computed-path imports documented as read once
- keep main's Deno refusal (#790), now naming Bun
- Bun MCP test opens a second session and gets the slot error
- docs, skill, --max-sessions help, AGENTS.md: bunx registration, one session on Bun, bun CI job
@github-actions github-actions Bot added the size:L Up to 499 changed lines, not counting tests, fixtures, translated docs, or the lockfile label Nov 7, 2026
@okwasniewski
okwasniewski requested a review from kdy1 November 7, 2026 11:49
@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web: 91 passed

All 91 tests in 41 files
Test Time
🟢 tests/aria-hidden-flow.e2e.ts · 2 passed 1.2s
🟢 aria hidden flow › picks the Pro plan by its visible text and confirms it 855ms
🟢 aria hidden flow › another plan shows the error and keeps the picker open 327ms
🟢 tests/async-states.e2e.ts · 2 passed 7.3s
🟢 async states › reads the code off the transient toast and verifies it once the button enables 2.4s
🟢 async states › a wrong code is rejected after the toast has gone 4.9s
🟢 tests/canvas-only.e2e.ts · 2 passed 496ms
🟢 canvas only › taps square, circle, then diamond at their pixel coordinates 289ms
🟢 canvas only › a shape out of order paints the error and resets progress 207ms
🟢 tests/canvas-whiteboard.e2e.ts · 2 passed 612ms
🟢 canvas whiteboard › drags both shapes into their slots and saves 358ms
🟢 canvas whiteboard › bounces a shape off the wrong slot and refuses to save early 254ms
🟢 tests/cart-totals.e2e.ts · 2 passed 549ms
🟢 cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug) 327ms
🟢 cart totals › a quantity never drops below one 222ms
🟢 tests/checkout-review.e2e.ts · 1 passed 257ms
🟢 shows a raw greeting token and an undefined delivery date (planted bug), yet places the order 257ms
🟢 tests/context-menu-trap.e2e.ts · 2 passed 795ms
🟢 context menu trap › right-click opens nothing; the row menu renames the file 419ms
🟢 context menu trap › the decoy menu items refuse and keep the file 376ms
🟢 tests/control-inventory.e2e.ts · 8 passed 4.6s
🟢 control inventory › check sets the checkbox and the radio without flipping a set one 414ms
🟢 control inventory › check picks a radio the pick replaces with its summary 311ms
🟢 control inventory › a long press and a double click are told apart from a tap 1.1s
🟢 control inventory › the file menu opens on right-click only and Rename renames the file 340ms
🟢 control inventory › setInputFiles attaches files named from the project root 235ms
🟢 control inventory › the details view is left through the browser history 306ms
🟢 control inventory › scrollIntoView brings the footnote into view 378ms
🟢 control inventory › every exercise done shows the success message 1.5s
🟢 tests/css-content-ui.e2e.ts · 2 passed 508ms
🟢 css content ui › the tile labelled Ship it advances and confirming ships 282ms
🟢 css content ui › a wrong tile shows the CSS-rendered error and keeps the picker 226ms
🟢 tests/date-picker.e2e.ts · 2 passed 891ms
🟢 date picker › navigates three months ahead and books the required date 511ms
🟢 date picker › confirming without the exact date shows the error 380ms
🟢 tests/debounced-search.e2e.ts · 2 passed 3.2s
🟢 debounced search › a decoy one character off is refused 1.5s
🟢 debounced search › picking the exact item adds it to the cart 1.6s
🟢 tests/deferred-dialog.e2e.ts · 1 passed 1.2s
🟢 one submit opens one confirm, even when double-clicked 1.2s
🟢 tests/div-soup.e2e.ts · 2 passed 968ms
🟢 div soup › a wrong code is rejected 441ms
🟢 div soup › the right code grants access after the delete tile fixes a typo 527ms
🟢 tests/drag-and-drop.e2e.ts · 2 passed 894ms
🟢 drag and drop › an out-of-order item resets the dropzone 466ms
🟢 drag and drop › dropping Banana then Cherry passes 428ms
🟢 tests/filter-deep-link.e2e.ts · 2 passed 399ms
🟢 filter deep link › filtering writes the query to the URL 208ms
🟢 filter deep link › opening a shared link restores the filter 191ms
🟢 tests/gift-card-purchase.e2e.ts · 3 passed 828ms
🟢 gift card purchase › refuses to buy without an amount 218ms
🟢 gift card purchase › refuses a recipient without an at sign 308ms
🟢 gift card purchase › sends the gift card 302ms
🟢 tests/hover-menu.e2e.ts · 2 passed 698ms
🟢 hover menu › a decoy leaf is refused 299ms
🟢 hover menu › hovering Account, then Billing, reaches Redeem voucher 399ms
🟢 tests/iframe-form.e2e.ts · 2 passed 943ms
🟢 iframe form › a wrong code is refused by the host page 453ms
🟢 iframe form › the code read from one frame applies through the other 490ms
🟢 tests/image-only-ui.e2e.ts · 3 passed 781ms
🟢 image only ui › the icons have no names, so the bare svg locator is ambiguous 204ms
🟢 image only ui › a wrong icon shows the error image and keeps the icons 271ms
🟢 image only ui › tapping the star, then the heart, verifies the icons 306ms
🟢 tests/infinite-scroll.e2e.ts · 1 passed 3.0s
🟢 scrolls the feed until the Golden Ticket loads, then claims it 3.0s
🟢 tests/kanban-board.e2e.ts · 3 passed 1.0s
🟢 kanban board › pointer drags move both cards into the goal layout 468ms
🟢 kanban board › a swipe along a path drags a card as well 247ms
🟢 kanban board › a plain tap moves nothing and the untouched board is rejected 285ms
🟢 tests/login-form.e2e.ts · 4 passed 1.2s
🟢 login form › rejects a malformed email 313ms
🟢 login form › rejects a short password 276ms
🟢 login form › rejects an unknown account 302ms
🟢 login form › signs in and logs out 329ms
🟢 tests/lying-labels.e2e.ts · 4 passed 1.1s
🟢 lying labels › the button whose accessible name says Pay now is the wrong one 232ms
🟢 lying labels › a text match for Pay now catches the decoy phrase too 188ms
🟢 lying labels › the field whose accessible name says Amount is the wrong one 343ms
🟢 lying labels › visible text drives the payment through 335ms
🟢 tests/native-dialogs.e2e.ts · 3 passed 755ms
🟢 native dialogs › answers the confirm, prompt, and alert chain 233ms
🟢 native dialogs › dismissing the confirm cancels the order 225ms
🟢 native dialogs › a wrong gift code is rejected 297ms
🟢 tests/newsletter-signup.e2e.ts · 1 passed 2.4s
🟢 subscribe renders no feedback and logs a TypeError (planted bug) 2.4s
🟢 tests/onboarding-wizard.e2e.ts · 5 passed 6.9s
🟢 onboarding wizard › back keeps what was typed and the brief comes back with it 434ms
🟢 onboarding wizard › the address step takes the brief through its bare-div dropdown 548ms
🟢 onboarding wizard › the profile step steers the calendar three decades back 1.7s
🟢 onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off 2.1s
🟢 onboarding wizard › the review step repeats the brief and finishes with the PIN 2.2s
🟢 tests/order-history.e2e.ts · 2 passed 1.8s
🟢 order history › overview and settings render their panels 389ms
🟢 order history › the orders tab never leaves its loading state (planted bug) 1.4s
🟢 tests/otp-auto-advance.e2e.ts · 3 passed 1.8s
🟢 otp auto-advance › focus advances with each digit and the full code verifies 645ms
🟢 otp auto-advance › backspace on an empty box clears and refocuses the previous one 248ms
🟢 otp auto-advance › a wrong code clears every box and starts over 935ms
🟢 tests/overlay-trap.e2e.ts · 1 passed 743ms
🟢 continues only after the banner and the overlay are gone 743ms
🟢 tests/playbook-cleanup.e2e.ts · 2 passed 807ms
🟢 playbook cleanup › deleting the draft removes it from an exact-name search 400ms
🟢 playbook cleanup › cancelling the dialog keeps the draft searchable 407ms
🟢 tests/price-sorting.e2e.ts · 3 passed 695ms
🟢 price sorting › opens in the featured order with sale prices struck through 210ms
🟢 price sorting › low to high sorts by the original price, not the paid one (planted bug) 210ms
🟢 price sorting › high to low is the same wrong order reversed (planted bug) 275ms
🟢 tests/product-reviews.e2e.ts · 1 passed 1.3s
🟢 the running-shoe page lists coffee-grinder reviews (planted bug) 1.3s
🟢 tests/promo-storefront.e2e.ts · 1 passed 1.4s
🟢 buys a deal once the promo chrome is out of the way 1.4s
🟢 tests/rich-text-editor.e2e.ts · 2 passed 809ms
🟢 rich text editor › bolding exactly the required word passes the check 374ms
🟢 rich text editor › bold that spills into the previous word fails the check 435ms
🟢 tests/saved-addresses.e2e.ts · 3 passed 1.0s
🟢 saved addresses › adds an address and clears the form 333ms
🟢 saved addresses › ignores an add with an empty field 308ms
🟢 saved addresses › delete shows its toast but keeps the address, on retry too (planted bug) 407ms
🟢 tests/shadow-dom-form.e2e.ts · 2 passed 642ms
🟢 shadow DOM form › rejects a wrong access code 332ms
🟢 shadow DOM form › grants access with the code from the hint inside the root 310ms
🟢 tests/smoke.e2e.ts · 1 passed 446ms
🟢 home lists the scenarios and opens one 446ms
🟢 tests/stale-dom.e2e.ts · 1 passed 2.3s
🟢 taps the moving target three times across rebuilds of the list 2.3s
🟢 tests/sticky-chrome.e2e.ts · 1 passed 365ms
🟢 accepts the terms once the button clears the sticky footer 365ms
🟢 tests/virtualized-table.e2e.ts · 1 passed 22.3s
🟢 scrolls the windowed table to the Golden Row and claims it 22.3s
🟢 tests/wishlist.e2e.ts · 2 passed 2.0s
🟢 wishlist › saving toggles the button and the tab badge 444ms
🟢 wishlist › the wishlist tab renders the empty state despite the badge count (planted bug) 1.6s

Tested by e2e 0.23.0 · 1m 23s · web · run artifacts

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e web agent: 33 passed

34 agent steps · 26 replayed from cache · 20 model calls · 87.8k tokens (85% cached) · $0.0054

All 33 tests in 5 files
Test Agent Time
🟢 tests-agent/control-inventory.e2e.ts · 7 passed 7 steps 9.6s
🟢 control inventory › check sets the checkbox and the radio 1 step 1.8s
🟢 control inventory › check picks a radio the pick replaces with its summary 1 step 469ms
🟢 control inventory › a long press and a double click are chosen over a plain tap 1 step 1.1s
🟢 control inventory › a right-click opens the file menu and Rename renames the file 1 step 2.8s
🟢 control inventory › upload attaches the named project files 1 step 2.6s
🟢 control inventory › back returns from the details view to the inventory 1 step 499ms
🟢 control inventory › scroll_to brings the footnote into view 1 step 436ms
🟢 tests-agent/login-form.e2e.ts · 1 passed 1 step 580ms
🟢 act signs in with the benchmark account 1 step 580ms
🟢 tests-agent/planted-bugs.e2e.ts · 3 passed 4 steps · 8 calls 15.9s
🟢 cart totals: a judgment catches the stale order summary 2 steps · 4 calls 8.9s
🟢 checkout review: a judgment catches the broken copy 1 step · 1 call 2.0s
🟢 wishlist: a judgment catches the lost saved items 1 step · 3 calls 5.0s
🟢 tests-agent/scenarios.e2e.ts · 21 passed 21 steps · 12 calls 1m 13s
🟢 act completes shadow-dom-form 1 step · 3 calls 4.3s
🟢 act completes lying-labels 1 step · 3 calls 4.4s
🟢 act completes iframe-form 1 step · 3 calls 4.5s
🟢 act completes infinite-scroll 1 step 3.7s
🟢 act completes virtualized-table 1 step 23.0s
🟢 act completes overlay-trap 1 step 1.0s
🟢 act completes sticky-chrome 1 step 970ms
🟢 act completes async-states 1 step · 3 calls 7.3s
🟢 act completes debounced-search 1 step 2.4s
🟢 act completes hover-menu 1 step 605ms
🟢 act completes drag-and-drop 1 step 496ms
🟢 act completes kanban-board 1 step 668ms
🟢 act completes context-menu-trap 1 step 2.7s
🟢 act completes stale-dom 1 step 526ms
🟢 act completes otp-auto-advance 1 step 764ms
🟢 act completes date-picker 1 step 890ms
🟢 act completes rich-text-editor 1 step 10.2s
🟢 act completes playbook-cleanup 1 step 667ms
🟢 act completes promo-storefront 1 step 1.3s
🟢 act completes gift-card-purchase 1 step 2.5s
🟢 act filters the list and the URL carries the filter 1 step 278ms
🟢 tests-agent/unique-cache.e2e.ts · 1 passed 1 step 361ms
🟢 act adds an address with a run-unique label 1 step 361ms

Tested by e2e 0.23.0 · 1m 40s · web · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 39 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread skills/e2e/references/mcp.md Resolved
Comment thread docs/reference/mcp.mdx Resolved
Comment thread docs/reference/runner.mdx Resolved
Comment thread docs/quickstart.mdx Resolved
Comment thread docs/ci.mdx Resolved
@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 12 replayed from cache · 60 model calls · 392.3k tokens (83% cached) · $0.0257

All 22 tests in 1 file
Test Agent Time
⏭️ tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped 25 steps · 60 calls 8m 17s
🟢 act completes Login Form 1 step 37.7s
🟢 act completes Infinite Scroll List 1 step 27.3s
🟢 act completes Modal Flow 1 step 11.6s
🟢 act completes Bottom Tabs 1 step 7.6s
🟢 act completes Text Input Variations 1 step 14.2s
🟢 act completes Broken Accessibility 2 steps · 5 calls 22.9s
🟢 act completes Vision Only 2 steps · 6 calls 28.7s
🟢 act completes Gestures 1 step · 7 calls 29.3s
🟢 act completes Async States 1 step · 10 calls 34.8s
🟢 act completes Debounced Search 1 step · 1 call 27.2s
🟢 act completes Flattened Registration Form 2 steps · 1 call 28.2s
🟢 act completes Flattened Login 2 steps · 11 calls 1m 0s
🟢 act completes Sticky Chrome Target 1 step 10.1s
🟢 act completes WebView Accessibility 1 step 12.1s
🟢 act completes Permission Prompt 1 step 11.2s
🟢 act completes Photo Picker 1 step · 2 calls 31.1s
🟢 act completes Product Catalog 1 step 10.1s
🟢 act completes Error Recovery 1 step 10.2s
🟢 act completes Choice Controls 1 step · 5 calls 33.3s
🟢 act completes Stripe PaymentSheet 1 step · 12 calls 35.7s
🟢 act completes Apple Pay 1 step 13.7s
⏭️ act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

Tested by e2e 0.23.0 · 8m 18s · ios-simulator · run artifacts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 7 files (changes from recent commits).

View guided diff | Turn on auto-fix | Re-trigger cubic

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android agent: 20 passed

24 agent steps · 15 replayed from cache · 39 model calls · 212.4k tokens (80% cached) · $0.0156

All 20 tests in 1 file
Test Agent Time
🟢 tests-agent/scenarios.e2e.ts · 20 passed 24 steps · 39 calls 5m 21s
🟢 act completes Login Form 1 step 30.4s
🟢 act completes Infinite Scroll List 1 step 22.8s
🟢 act completes Modal Flow 1 step 12.1s
🟢 act completes Bottom Tabs 1 step 7.0s
🟢 act completes Text Input Variations 1 step 6.3s
🟢 act completes Broken Accessibility 2 steps · 5 calls 14.8s
🟢 act completes Vision Only 2 steps · 1 call 6.2s
🟢 act completes Gestures 1 step · 9 calls 22.9s
🟢 act completes Async States 1 step · 11 calls 26.8s
🟢 act completes Debounced Search 1 step 7.2s
🟢 act completes Huge Virtualized List 1 step 1m 11s
🟢 act completes Flattened Registration Form 2 steps · 1 call 9.5s
🟢 act completes Flattened Login 2 steps · 6 calls 16.4s
🟢 act completes Sticky Chrome Target 1 step 7.4s
🟢 act completes WebView Accessibility 1 step · 6 calls 15.8s
🟢 act completes Permission Prompt 1 step 7.0s
🟢 act completes Product Catalog 1 step 8.5s
🟢 act completes Error Recovery 1 step 9.8s
🟢 act completes Choice Controls 1 step 9.0s
🟢 act completes Sequential Onboarding 1 step 10.2s

Tested by e2e 0.23.0 · 5m 22s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e android: 1 flaky, 56 passed, 1 skipped

⚠️ 1 flaky test passed on a retry

⚠️ control inventory › a label query resolves a labeled text field
tests/fixtures.ts:78

ACTION_FAILED (beforeEach) at step 3 of 3: expect.toBeVisible getByLabel("Control Inventory. "), after 5.0s

operation timed out: getByLabel("Control Inventory. ")

  • Screen: dev.e2e.benchmark

Evidence: screenshot, log

All 58 tests in 7 files
Test Time
⚠️ tests/controls.e2e.ts · 1 flaky, 23 passed, 1 skipped 2m 47s
⚠️ control inventory › a label query resolves a labeled text field (1 failed attempt first) 7.7s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 15.0s
🟢 control inventory › a placeholder query and attribute reads find the field 10.5s
🟢 control inventory › a secure field takes a fill and denies value reads 8.5s
🟢 control inventory › press sends a character and Enter through the soft keyboard 8.9s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 9.1s
🟢 control inventory › toBeFocused follows the focus 9.1s
🟢 control inventory › a switch checks, unchecks, and reports its state 9.1s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 7.1s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 10.5s
🟢 control inventory › a tab role answers getByRole 4.2s
🟢 control inventory › a radio group reports the checked option 7.7s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 6.8s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 6.9s
🟢 control inventory › a header answers its accessible name 4.0s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 5.6s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 5.6s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 7.2s
🟢 control inventory › a double tap lands inside the 300 ms window 6.6s
🟢 control inventory › an ambiguous locator fails at once 5.2s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 8.7s
🟢 refusals › verbs a device never declares are refused before the engine 6ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 1.6s
🟢 refusals › a missing locator fails as not found once its timeout passes 1.1s
🟢 tests/device-fixture.e2e.ts · 10 passed 1m 23s
🟢 device fixture › setOrientation rotates the window the app measures 5.0s
🟢 device fixture › home leaves the app, openApp brings it back where it was 5.1s
🟢 device fixture › foregroundApp sees the home screen after home() 3.9s
🟢 device fixture › closeApp then openApp relaunches on the home list 7.6s
🟢 device fixture › closeApp leaves the next commands on the same device 7.0s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 4.6s
🟢 device fixture › device.back pops the scenario 4.8s
🟢 device fixture › unstable_raw reaches the test session through agent-device 3.9s
🟢 device fixture › setNetwork and setAirplaneMode change what the app sees 16.0s
🟢 device fixture › setLocation and clearLocation change what the app reads 25.4s
🟢 tests/device.e2e.ts · 5 passed 49.5s
🟢 permission prompt › allowing the system dialog enables the microphone 6.9s
🟢 permission prompt › a denied prompt recovers once the permission is granted outside 15.0s
🟢 sequential onboarding echoes the exact values 11.2s
🟢 onboarding rejects a malformed email before advancing 7.1s
🟢 dismissing the keyboard over choice controls leaves them as they were 9.4s
🟢 tests/flows-lists.e2e.ts · 7 passed 2m 26s
🟢 infinite scroll reaches the target item 22.3s
🟢 product catalog adds exactly two of the right variant 10.5s
🟢 gestures: long-press, then swipe left 8.5s
🟢 gestures: double-tap completes the flow 10.1s
🟢 sticky chrome: accept below the fold, then continue 11.1s
🟢 async states: load, pull to refresh, claim through the toast 12.4s
🟢 huge virtualized list reaches row 512 1m 11s
🟢 tests/flows.e2e.ts · 7 passed 56.3s
🟢 text inputs unlock the submit button 8.0s
🟢 filled text inputs report their values 6.3s
🟢 modal flow confirms through the native alert 10.0s
🟢 bottom tabs act inside the last tab 7.4s
🟢 error recovery retries, then confirms the delete 9.1s
🟢 choice controls place the exact order 6.5s
🟢 debounced search selects the target once results arrive 9.1s
🟢 tests/login-form.e2e.ts · 3 passed 28.0s
🟢 login form › rejects a malformed email 7.7s
🟢 login form › rejects an unknown account 7.7s
🟢 login form › signs in and logs out 12.5s
🟢 tests/smoke.e2e.ts · 1 passed 6.0s
🟢 home lists the scenarios and opens one 6.0s

Tested by e2e 0.23.0 · 9m 29s · android-emulator · run artifacts

@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e ios: 55 passed, 1 skipped

All 56 tests in 7 files
Test Time
⏭️ tests/controls.e2e.ts · 23 passed, 1 skipped 2m 54s
🟢 control inventory › a label query resolves a labeled text field 10.2s
🟢 control inventory › a labeled field answers name, display value, value reads, fill, and clear 14.8s
🟢 control inventory › a placeholder query and attribute reads find the field 4.7s
🟢 control inventory › a secure field takes a fill and denies value reads 8.2s
🟢 control inventory › press sends a character and Enter through the soft keyboard 9.6s
🟢 control inventory › a field autofocuses on mount and focus() moves the focus 9.9s
🟢 control inventory › toBeFocused follows the focus 8.9s
🟢 control inventory › a switch checks, unchecks, and reports its state 9.4s
🟢 control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap 7.9s
🟢 control inventory › checkbox and radio roles check, uncheck, and report their state 12.0s
🟢 control inventory › a radio group reports the checked option 8.5s
🟢 control inventory › buttons report disabled, enabled, and a state that arrives later 7.5s
🟢 control inventory › a display: none element is absent rather than hidden, its visible twin attached 7.1s
🟢 control inventory › a header answers its accessible name 4.1s
🟢 control inventory › a tab row answers count, all, texts, filters, positions, and switching 6.1s
🟢 control inventory › a list answers counts, list-form text, reads, and boxes in order 7.4s
⏭️ control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢 control inventory › a long press lands as one 8.3s
🟢 control inventory › pointer verbs land at the point asked for 10.1s
🟢 control inventory › an ambiguous locator fails at once 6.4s
🟢 control inventory › screenshot, restart, and clearState reopen the app on its home list 11.4s
🟢 refusals › verbs a device never declares are refused before the engine 2ms
🟢 refusals › keys the soft keyboard cannot send are refused by the engine 512ms
🟢 refusals › a missing locator fails as not found once its timeout passes 1.2s
🟢 tests/device-fixture.e2e.ts · 11 passed 1m 28s
🟢 device fixture › setAppearance flips the color scheme the app reads 13.1s
🟢 device fixture › setOrientation rotates the window the app measures 8.7s
🟢 device fixture › the clipboard reads back what was written 6.1s
🟢 device fixture › home leaves the app, openApp brings it back where it was 6.5s
🟢 device fixture › closeApp then openApp relaunches on the home list 7.8s
🟢 device fixture › closeApp leaves the next commands on the same device 11.6s
🟢 device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN 5.2s
🟢 device fixture › device.back pops the scenario 5.4s
🟢 device fixture › unstable_raw reaches the test session through agent-device 4.3s
🟢 device fixture › setLocation and clearLocation change what the app reads 10.1s
🟢 device fixture › enrollBiometrics and setBiometrics answer a biometric prompt 9.6s
🟢 tests/device.e2e.ts · 3 passed 1m 7s
🟢 sequential onboarding echoes the exact values 40.4s
🟢 onboarding rejects a malformed email before advancing 14.5s
🟢 dismissing the keyboard over choice controls leaves them as they were 11.8s
🟢 tests/flows-lists.e2e.ts · 7 passed 3m 19s
🟢 infinite scroll reaches the target item 25.6s
🟢 product catalog adds exactly two of the right variant 10.1s
🟢 gestures: long-press, then swipe left 9.7s
🟢 sticky chrome: accept below the fold, then continue 10.5s
🟢 async states: load, pull to refresh, claim through the toast 10.8s
🟢 huge virtualized list reaches row 512 2m 0s
🟢 web view coupon form applies the code shown on the page 12.0s
🟢 tests/flows.e2e.ts · 7 passed 1m 11s
🟢 text inputs unlock the submit button 13.3s
🟢 filled text inputs report their values 11.1s
🟢 modal flow confirms through the native alert 9.7s
🟢 bottom tabs act inside the last tab 7.4s
🟢 error recovery retries, then confirms the delete 9.3s
🟢 choice controls place the exact order 8.1s
🟢 debounced search selects the target once results arrive 11.9s
🟢 tests/login-form.e2e.ts · 3 passed 50.3s
🟢 login form › rejects a malformed email 14.5s
🟢 login form › rejects an unknown account 14.6s
🟢 login form › signs in and logs out 21.2s
🟢 tests/smoke.e2e.ts · 1 passed 8.1s
🟢 home lists the scenarios and opens one 8.1s

Tested by e2e 0.23.0 · 5m 54s · ios-simulator · run artifacts

@okwasniewski

Copy link
Copy Markdown
Member Author

[claude-opus-5-5] responding on behalf of okwasniewski

Everything is green except web agent and the benchmark gate behind it: act completes infinite-scroll fails with REPLAY_STALE (action-failed). It failed the same way on main three times today, before this PR ran (runs 42362410220, 42369390756, 42369507311), so its committed recording is stale on main. Re-recording it needs a live model run, so it is left for a separate fix on main.

Two flaky tests failed once here and passed on rerun. Neither is touched by this PR:

  • failure-evidence.test.ts asserts waitedMs >= 300 and got 299: the locator starts its clock (packages/e2e/src/locator/engine.ts:248) about 1 ms after the action's 300 ms deadline is created.
  • @e2e-dev/web screenshot-navigation.test.ts: page.screenshot: Execution context was destroyed. It also failed on main at 10:16, after the playwright-core 1.65.0 bump.

@okwasniewski
okwasniewski merged commit 5847701 into main Nov 9, 2026
30 checks passed
@okwasniewski
okwasniewski deleted the oskar/bun-runtime-support branch November 9, 2026 17:25
@github-actions github-actions Bot mentioned this pull request Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:L Up to 499 changed lines, not counting tests, fixtures, translated docs, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant